Task Space Tile Coding: In-Task and Cross-Task Generalization in Reinforcement Learning

Exploiting the structure of a domain is an important prerequisite for being able to efficiently use reinforcement learning in larger state spaces. In this paper, we show how to benefit from the explicit representation of structural features in so-called structure space aspectualizable state spaces. We introduce task space tile coding as a mechanism to achieve generalization over states with identical structural properties. This leads to a significant improvement of learning performance. Policies learned with task space tile coding can also be applied to unknown environments sharing the same structure space and thus enable for a faster learning in new tasks.